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The  thinned  response  of  an  edge  detector  constitutes 
a  set  of  edge-points  lying  along  edges  in  the  original 
image.  It  is  possible  to  link  each  edge-point  to  its  appro¬ 
priate  neighbor  on  either  side  and  thus  delineate  these 
edges  in  the  image.  This  is  accomplished  by  considering 
all  contours  produced  by  thresholding  which  pass  through  a 
given  edge-point.  For  each  such  contour,  the  edge-point 
nearest  the  given  edge-point  along  the  contour  in  the  clock¬ 
wise  direction  is  recorded.  The  edge-point  appearing  most 
often  as  clockwise  associate  to  the  given  edge-point  is  then 
assigned  as  the  clockwise  neighbor.  A  figure  of  merit  based 
on  distance,  straightness  and  contrast  is  used  to  break  any 
ties.  The  counter-clockwise  neighbor  is  computed  similarly. 
The  resulting  weighted  directed  graph  is  available  for  seg¬ 
mentation  into  long  chains,  traversal,  line-fitting  or  tem¬ 
plate  matching.  The  use  of  contours  to  propose  pairings  of 
edge-points  is  an  example  of  the  power  of  convergent  evidence 
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1.  Introduction 


The  importance  of  edge  description  is  well  documented  by 
a  rich  literature.  A  large  segment  of  the  literature  concerns 
the  detection  of  points  on  region  boundaries  and  the  measure¬ 
ment  of  features,  such  as  magnitude  and  direction,  at  those 
points.  For  a  survey  of  edge  detection  techniques,  see  [1]. 
The  edge  points  and  feature  measurements  resulting  from  edge 
detection  are  put  to  many  uses  including  threshold  determin¬ 
ation,  segmentation,  image  matching,  etc. 

The  grouping  of  edge  points  into  higher  order  entities 
such  as  lines  or  curves  has  also  received  a  good  deal  of 
attention.  For  an  overview,  see  Section  8.4  of  Rosenfeld  [2]. 
Iannino  and  Shapiro  [3]  survey  the  Hough  transform  approach  in 
which  collinear  points  form  detectable  clusters  and  are  thus 
associated  into  line  segments.  The  sequential  approach 
(tracking)  attempts  to  extend  the  current  line  by  affixing  the 
best  available  edge  point.  Methods  of  this  type  are  described 
by  Montanari  14],  Martelli  [5]  and  Ashkar  and  Modestino  [6]. 

A  third  class  of  methods  is  parallel  in  nature  using,  e.g., 
directed  propagation  to  fill  small  gaps  or  relaxation  to  ad¬ 
just  incorrectly  labelled  edge  points.  See,  for  example, 
Zucker ,  et  al .  [ 7] . 

One  may  consider  the  problem  of  grouping  edge  points  in  a 
more  general  light.  We  might  wish  to  group  together  those 
points  which  bound  the  same  region  in  an  image.  However,  the 


notion  of  "region"  is  imprecise  due  to  conditions  of  poor 
lighting,  shadows,  non-planar  surfaces,  etc.  If  by  a  "region" 
we  mean  a  "thresholdable  region"  then  we  may  group  together 
those  edge  points  lying  on  the  same  contour  after  thresholding. 
This  is  the  approach  taken  by  Nakagawa  and  Rosenfeld  [8], 

However  their  pictorial  results  show  that  wrong  associations 
are  made  when  the  assumption  of  region  thresholdability  is 
violated.  The  problem  remains  to  associate  edge  points  without 
requiring  that  the  adjacent  regions  be  thresholdable. 

In  [9] ,  the  author  showed  that  the  coincidence  of  edge 
points  with  region  boundaries  can  serve  as  evidence  for  the 
presence  of  an  object.  In  that  method  (called  Superslice) ,  an 
object  might  be  evident  over  a  range  of  thresholds  and,  for 
each  threshold,  might  be  represented  by  a  different  contour. 
Superslice  selects  the  contour  with  the  greatest  percentage  of 
coincident  edge  points.  That  approach  relies  on  the  convergence 
of  evidence  from  two  sources,  thresholding  and  edge  detection, 
to  perform  region  extraction.  The  principle  of  convergent 
evidence  is  utilized  in  the  current  work  to  link  each  edge  point 
to  its  best  associate  in  the  clockwise  and  counterclockwise 
direction.  The  algorithm  which  accomplishes  this  is  called 
Superlink. 


2.  Method 


To  restate  the  problem:  wo  are  given  a  set  of  pixels 
(edge-points)  corresponding  to  the  locations  and  values  of 
significant  edge  maxima  in  an  image.  Assuming  that  edge- 
points  lie  on  edges  extending  some  distance  on  either  side, 
we  wish  to  associate  each  edge-point  to  the  appropriate 
edge-point  on  either  side. 

The  solution  is  as  follows:  Let  E  be  the  set  of  edge- 
points  and  let  etE.  Suppose  at  some  threshold  t,  there  is 
a  connected  component  of  above- threshold  points  whose  bound¬ 
ary  includes  e.  (We  call  all  such  boundaries  "contours.") 

Let  e=e^ , , . . . , e^ , en+^=e  be  the  succession  of  edge-points 
encountered  in  a  clockwise  traversal  of  the  contour.  We 
define  C(e,x)=e2  (CC(e,x)=e^)  as  the  clockwise  (counterclock¬ 
wise)  neighbor  of  e.  Each  neighbor  C(e,x)  delimits  a  path 
from  e  to  C(e,x)  along  a  contour.  At  a  different  threshold 
x',  C(e,x')  might  delimit  a  different  path.  We  can  compare 
paths  preferring  some  to  others  based  on  various  features  and 
compute  for  each  path  a  figure  of  merit.  Thus,  for  example, 
short,  straight  paths  are  preferred  as  are  those  whose  contrast 
does  not  vary  much  from  one  end  to  the  other.  The  figure  of 
merit  used  here  is  a  weighted  linear  combination  of  length, 
straightness  and  contrast,  although  we  recognize  that  many 
other  possibilities  and  combinations  exist.  If  no  contour 


passes  through  e  at  a  given  threshold  t  then  C(e,x),  CC(e,x) 
are  undefined. 

Consider  the  collection  (including  duplicates)  of  clock¬ 
wise  neighbors  C  (e, x^) , . . . ,C (e, x^)  for  some  set  of  thresholds 
T= {x^,...,x^}.  Define  C(e)  to  be  the  neighbor  of  e  occurring 
most  often  in  the  collection.  Thus  C(e)  is  chosen  as  the 
clockwise  associate  of  e  (the  counterclockwise  associate  CC(e) 
is  defined  similarly) .  In  the  event  that  several  edge-points 
occur  equally  often,  choose  as  associate  that  edge-point  con¬ 
tender  with  the  highest  figure  of  merit.  It  makes  sense  to 
delete  any  associate  whose  figure  of  merit  is  below  some 
threshold,  indicating  the  weakness  of  the  evidence  for  linking 
the  edge  points. 

When  completed,  the  process  has  selected  for  each  edge- 
point  e  (at  most)  one  clockwise  associate  C(e)  and  (at  most) 
one  counterclockwise  associate  and  has  computed  their  figures 
of  merit.  Note  however  that  the  association  is  not  necessarily 
mutual  (symmetric),  i.e.,  it  is  not  true  that  CC(C(e))=e  or 
that  C(CC(e))=e.  This  is  reasonable  since  it  is  possible  for 
an  edge-point  to  be  in  the  vicinity  of  a  corner  at  which  three 
or  more  surfaces  meet.  It  may  also  result  from  breaking  ties 
or  from  edge-point  clustering.  Nonetheless,  the  great  majority 
of  linkings  do  turn  out  to  be  mutual,  providing  additional 
evidence  of  their  correctness. 


3.  Implementation 


Superlink  has  been  implemented  on  the  Univac  1108  (Fxec  8) 
and  the  PDP-11/45  (UNIX)  as  a  sequence  of  modules  described 
below  (Fiqure  1).  In  the  first  step,  edqe-points  are  located 
in  the  input  image  by  thinning  the  response  of  an  edge  detector. 
The  detector  we  used  computed  the  horizontal  and  vertical  dif¬ 
ferences  of  2x2  averages.  The  resulting  difference  images  were 
separately  thinned  by  local  non-maximum  suppression.  The  edge- 
point  image  results  from  taking  the  maximum  of  the  thinned 
horizontal  and  vortical  responses  and  deleting  all  insignifi¬ 
cant  edge  responses  (a.1). 

Prior  to  contour  extraction,  it  is  necessary  to  determine 
the  set  of  gray  levels  T  at  which  to  threshold  the  input  image. 
Naturally,  T  can  consist  of  the  whole  gray-level  range  in  the 
input  image.  However,  this  can  be  expensive.  Frequently,  one 
has  knowledge  of  the  likely  gray  level  range  of  the  edge-points 
of  interest.  For  example,  in  a  two  population  image  (object/ 
background)  the  range  between  the  modes  would  define  T.  Alter¬ 
natively,  one  could  sample  the  gray  level  range  choosing  every 
other  gray  level,  etc.  The  danger  in  any  scheme  which  skips 
gray  levels  is  that  all  contours  at  the  ignored  gray  level 
threshold:  as  well  as  the  possible  pairings  of  edge-points 
along  these  contours  are  lost.  Thus  less  evidence  is  available 
when  choosing  associates,  which  makes  the  selections  more  de¬ 
pendent  on  figure  of  merit  (a  distinctly  weaker  criterion  than 
"most  often  occurring  edge-point").  Nonetheless,  the  degradation 


incurred  by  deleting  gray  levels  is  gradual  as  is  discussed 
in  the  next  section. 


Once  a  set  of  gray  level  thresholds  is  chosen,  the  con¬ 
tours  are  extracted  (10]  and  stored  in  Freeman  chain  code. 

This  takes  one  pass  over  both  images  (gray  level  and  edge- 
point)  for  each  of  the  thresholds.  The  accumulated  chain- 
encoded  contours  are  stored  on  disk.  Next,  the  disk  file  is 
read  contour  by  contour.  For  each  contour,  the  sequence  of 
edge-points  is  noted  and  the  figure  of  merit  is  computed  for 
each  adjacent  pair  in  the  sequence.  The  coordinates  of  each 
pair  of  edge-points  and  its  figure  of  merit  are  then  written 

to  a  file.  The  file  of  edge-point  pairs  is  quite  large 

2 

(40,000  pairs  for  a  256  image  using  12  thresholds).  It  is 
sorted  (using  a  system  sorting  package)  so  that  all  pairs 
containing  a  given  edge-point  are  in  contiguous  sequence. 

The  sorted  file  is  a  sequence  of  edge  point  lists.  Each 
edge-point  list  is  the  set  of  pairs  for  a  given  edge-point. 
Once  it  is  read,  it  is  straightforward  to  compute  the  most 
numerous  associate  or,  in  the  event  of  a  tie,  the  best  figure 
of  merit.  The  (edge-point,  associate)  pair  is  then  written 
to  a  separate  file.  Finally,  the  associates  file  is  converted 
to  an  image  by  taking  each  associated  pair  of  coordinate  point 
and  drawing  a  straight  line  in  the  image  to  signify  their 
linkage.  This  last  step  is  convenient  for  display  purposes: 

. 

L  _ _ _ 


however,  the  use  of  a  straight  line  to  join  edge-points  only 
serves  as  an  approximation  to  the  contour  segment  which 
actually  bridges  the  two  points. 


4.  Results 


The  algorithm  as  described  in  the  previous  section  has 
been  run  using  a  variety  of  input  images.  Figure  2  shows 
several  FLIR  images  of  military  vehicles  and  illustrates  the 
extracted  edge-points.  Figure  3  displays  the  edge-points  and 
their  links.  Links  whose  figures  of  merit  were  below  a  preset 
threshold  are  not  shown.  Our  experience  has  been  that  select¬ 
ing  a  threshold  for  the  figure  of  merit  is  difficult  unless  a 
very  generous  one  is  used,  as  was  done  here.  Normally,  all 
but  about  3%  of  the  proposed  links  appear  to  be  justifiable. 

Of  course,  some  links  are  the  result  of  more  evidence  than 
others  and  the  figure  of  merit  attempts  to  capture  this.  Thus 
the  underlying  data  structure  is  a  weighted  directed  graph,  with 
the  figure  of  merit  corresponding  to  the  weight. 

A  portion  of  an  image  of  automotive  parts  from  a  GM  data 
base  [11]  shows  the  effect  of  thresholding  the  figure  of  merit 
(Figure  4).  As  more  links  are  deleted,  some  "obviously  correct" 
linkages  disappear  while  others  which  are  somewhat  more  dubious 
remain.  A  blow-up  of  the  upper-right  portion  (Figure  5)  shows 
that  where  the  edge-points  form  a  staircase  pattern,  the  link¬ 
ages  form  small  loops.  Small  loops  may  also  result  from  the 
linkages  of  isolated  points.  This  demonstrates  as  well  that 
the  process  which  creates  edge-points  must  locate  the  points 
accurately,  thin  them  sufficiently,  and  discard  those  deemed 
not  to  correspond  to  actual  edges.  Figure  6  shows  the  effect 


of  thresholding  the  edge-point  population  on  the  linkages 
produced. 

It  was  mentioned  previously  the*:,  the  most  effective  link¬ 
ages  are  produced  when  all  gray  level  thresholds  are  employed 
but  that  degradation  is  graceful  as  gray  levels  are  omitted. 
Figure  7  illustrates  the  effect  of  retaining  only  every  other 
gray  level.  Figure  8  shows  another  example  of  the  GM  data 
base  along  with  the  resulting  linkages  based  on  every  other 
gray  level. 


5.  Conclusions 


The  Superlink  algorithm  joins  edge-points  based  on 
thresholding  evidence.  By  and  large,  its  proposed  linkages 
are  reasonable.  Much  work,  however,  remains.  First,  the 
current  figure  of  merit,  while  well-founded,  is  heuristic  and 
could  benefit  from  further  analysis.  For  example,  no  notice 
is  taken  currently  of  mutual  linkages;  yet,  clearly,  this  is 
powerful  evidence  that  the  linkage  is  legitimate.  Secondly, 
the  choice  of  edge  points  depends  on  the  type  of  edge  detector, 
the  method  of  thinning  and  the  elimination  of  noise  points. 
Third,  the  steps  making  up  Superlink  can  be  consolidated  and 
the  processes  made  to  run  much  more  efficiently.  Finally, 
new  algorithms  are  needed  to  track  the  linkage  data  structure 
and  to  extract  consistent  boundaries. 
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Figure  1. 


Superlink  process i mi  steps 
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Figure  4.  Effects  of  thresholding  the  figure  of  merit 

a.  Original  gray  level  window 

b.  Edge-point  image 

c.  Edge-point  associates,  threshold  at  .4 

d.  Same  as  c. ,  threshold  at  .5 

e.  Same  as  c.,  threshold  at  .6 


Figure  3. 


Figure  5. 


Blowup  of  upper-right  corner  of  Figurt  4c 
to  display  the  effects  of  thinning  errors 


Figure  6.  The  effects  of  edge-point  selection 

a.  Edge  point  subset  of  Figure  4b. 

b.  Edge  point  associates  for  upper-right  quadrant 


Figure  7.  The  effect  of  thresholding  at  even  gray 
levels  only.  Compare  with  Figure  5. 
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Figure  8.  Another  image  from  the  C.M  Pat  a  Base 

a.  Original 

b .  Edg e- po i n  t  imaq e 

c.  Edqe-point  associations 
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